Database generation from natural language text documents

ABSTRACT

Some embodiments may perform operations of a process that includes obtaining a natural language text document and use a machine learning model to generate a set of attributes based on a set of machine-learning-model-generated classifications in the document. The process may include performing hierarchical data extraction operations to populate the attributes, where different machine learning models may be used in sequence. The process may include using a pre-trained Bidirectional Encoder Representations from Transformers (BERT) model augmented with a pooling operation to determine a BERT output via a multi-channel transformer model to generate vectors on a per-sentence level or other per-text-section level. The process may include using a finer-grain model to extract quantitative or categorical values of interest, where the context of the per-sentence level may be retained for the finer-grain model.

RELATED APPLICATIONS

This application claims benefit of U.S. Provisional Patent ApplicationNo. 63/227,793 filed Jul. 30, 2021 and titled “DATABASE GENERATION FROMNATURAL LANGUAGE TEXT DOCUMENTS”, U.S. Provisional Patent ApplicationNo. 63/227,790 filed Jul. 30, 2021 and titled “DATABASE QUERY GENERATIONUSING NATURAL LANGUAGE TEXT”, and U.S. Provisional Patent ApplicationNo. 63/227,796 filed Jul. 30, 2021 and titled “SELF-EXECUTING PROTOCOLGENERATION FROM NATURAL LANGUAGE TEXT”, the entireties of which are eachhereby incorporated by reference.

This application is related to co-pending U.S. patent application titled“DATABASE QUERY GENERATION USING NATURAL LANGUAGE TEXT” filed byApplicant on Jul. 29, 2022 (Attorney Docket No. 062006-0569758), and toco-pending U.S. patent application titled “SELF-EXECUTING PROTOCOLGENERATION FROM NATURAL LANGUAGE TEXT” filed by Applicant on Jul. 29,2022 (Attorney Docket No. 062006-0569760), the entireties of which areeach hereby incorporated by reference.

BACKGROUND

Natural language processing includes techniques usable to processdocuments and discern meaning from the documents. Applications oftenemploy natural language processing techniques to perform documentprocessing tasks, such as translation, information extraction,summarization, text generation, or document assessment. Natural languageprocessing often includes training a model to analyze natural languagetext and employing the trained model to determine one or more resultsbased on the natural language text.

SUMMARY

The following is a non-exhaustive listing of some aspects of the presenttechniques. These and other aspects are described in the followingdisclosure.

Provided in some embodiments are techniques that include methods andsystems for discerning information from text documents. In someembodiments, an artificial intelligence (AI) model, such as a deeplearning type of machine learning model, is employed to generate a setof attributes of a natural language text document based on a set ofmachine-learning-model-generated classifications of the document. Theprocess may include, for example, using different deep learning modelsin sequence to perform hierarchical data extraction operations topopulate attributes. In some embodiments, the process includes using amulti-channel transformer model. For example, the process may includeusing a multi-channel transformer model, such as a pre-trainedBidirectional Encoder Representations from Transformers (BERT) modelaugmented with a pooling operation to determine a BERT output togenerate vectors on a per-sentence level or other per-text-sectionlevel. A finer-grain model may, for example, be used to extractquantitative or categorical values of interest, where the context of theper-sentence level may be retained for the finer-grain model.

Provided in some embodiments is a tangible, non-transitory,machine-readable medium storing instructions that are executable by adata processing apparatus to cause a data processing apparatus toperform operations described herein, such as those described above.

Provided in some embodiments is a system, including one or moreprocessors and memory storing instructions that are executable by a dataprocessing apparatus to cause a data processing apparatus to performoperations described here, such as those described above.

BRIEF DESCRIPTION OF THE DRAWINGS

The above-mentioned aspects and other aspects of the present techniqueswill be better understood when the present application is read in viewof the following figures in which like numbers indicate similar oridentical elements:

FIG. 1 is a schematic diagram that illustrates a first computingenvironment in accordance with one or more embodiments of the presenttechnique.

FIG. 2 is a diagram that includes a logic-and-physical-architecturesystem in accordance with one or more embodiments of the presenttechnique.

FIG. 3 is a flowchart diagram that illustrates a method of training amachine learning model for information extraction in accordance with oneor more embodiments of the present technique.

FIG. 4 is a flowchart diagram that illustrates a method of extractinginformation of a document in accordance with one or more embodiments ofthe present technique.

FIG. 5 is a flowchart diagram that illustrates a method employingmicro-ai-models in accordance with one or more embodiments of thepresent technique.

FIG. 6 is a flowchart diagram that illustrates employing micro-ai-modelsin accordance with one or more embodiments of the present technique.

FIG. 7 is a diagram that illustrates an example computing device inaccordance with one or more embodiments of the present technique.

While the present techniques are susceptible to various modificationsand alternative forms, specific embodiments are shown by way of examplein the drawings and are described in detail. The drawings may not be toscale. The drawings and detailed description are not intended to limitthe present techniques to the particular form described, but areintended to cover all modifications, equivalents, and alternativesfalling within the spirit and scope of the present techniques as definedby the appended claims.

DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS

To mitigate the problems described herein, the inventors had to bothinvent solutions and, in some cases just as importantly, recognizeproblems overlooked (or not yet foreseen) by others in the field ofnatural language processing. Indeed, the inventors wish to emphasize thedifficulty of recognizing those problems that are nascent and willbecome much more apparent in the future should trends in industrycontinue as the inventors expect. Further, because multiple problems areaddressed, some embodiments are problem-specific, and not allembodiments address every problem with traditional systems describedherein or provide every benefit described herein. That said,improvements that solve various permutations of these problems aredescribed below.

Information stored in unstructured natural language text is often storedin database systems to increase data retrieval efficiency and to providefor cross-document analysis of data stored in unstructured documents.Unfortunately, the conversion of this information into database valuesis often tedious and difficult due to the variations in terminology,document structure, and priorities of the documents. Changes in databasetable sizes, misalignments in characters, spelling errors, or the use ofunconventional terminology often results in failed or incorrectinterpretations of values. These and other challenges often frustrateoperations attempting to correctly extract values from a set ofdocuments.

As described herein, certain embodiments provide perform operations toextract information from a document using a set of artificialintelligence (AI) learning models (or “learning models”), such as deeplearning models or other machine learning models, and to store theinformation in a structured database. The information may be prioritizedbased on predefined or dynamically defined categories. In someembodiments, training the set of learning models may include usingself-learning models or semi-automated learning models. Some embodimentsmay further train one or more learning models to store the extractedinformation in a structure that is organized based on the document.Furthermore, some embodiments may use the set of learning models in ahierarchical arrangement and further dynamically structure a userinterface associated with model training to increase trainingefficiency.

Example Systems

FIG. 1 is a schematic diagram that illustrates a first computingenvironment (or “system”) 100 in accordance with one or moreembodiments. In the illustrated embodiment, the system 100 includes acomputer system 102 (e.g., including a classifier subsystem 110 and adata extraction subsystem 112), a set of user devices 104 (e.g.,including user devices 104 a-104 c) and databases 132 (e.g., includingdocument databases 134, a model database 136 and a document featuredatabase 138) communicatively coupled by way of a network 150.

Although certain embodiments are described herein as being performed byparticular components of computer system 102 for the sake of example,some or all of those operations may, in some embodiments, be performedby other components of computer system 102 or other components of system100. As an example, while one or more operations are described herein asbeing performed by components of the computer system 102, thoseoperations may, in some embodiments, be performed by components of theset of user devices 104. Various instructions, messages, or other setsof values may be communicated between components of the system 100 via anetwork 150.

In some embodiments, the user devices 104 are mobile or fixed terminals.For example, the user devices 104 a-104 c may each be a desktopcomputer, a notebook computer, a tablet computer, a smartphone, awearable device, or another user device. Users may, for example, utilizeone or more user devices 104 a-104 c to interact with one another, withone or more servers, or with other components of system 100.

In some embodiments, data is obtained from one or more databases 132 fortraining or use by a set of AI learning models (e.g., deep learningmodels or other types of machine learning models) to process documents.Machine learning models may be a type of AI learning models, and deeplearning models may, for example, be a type of machine model thatemploys a neural network with multiple layers (which may include one ormore hidden layers). Although certain embodiments are described asemploying one or more deep learning models for the purpose ofillustrations, certain embodiments may employ other suitable types ofmodels, such as other AI and machine learning models. Trainingoperations for a deep learning model may include, for example, obtainingcorpora from a document database 134 and using the corpora to determinemodel parameters for the deep learning model. The model parametersdetermined may then be stored in the model database 136 and retrievedfor use in classifying a document or determining document feature values(or “field values”), which may be stored in the document featuredatabase 138. In some embodiments, a set of document features (or“fields”) for a document are retrieved from a document feature database138 based on a set of categories assigned to the document.

In some embodiments, the classifier subsystem 110 is employed toclassify a document and the data extraction subsystem 112 is used todetermine features and values of features contained in a document. Forexample, the classifier subsystem 110 may use a first set of modelparameters from the model database 136 for a first deep learning modelto classify a document to determine a set of document features of adocument, and the data extraction subsystem 112 may use a second set ofmodel parameters from the model database 136 for a second deep learningmodel to determine values associated with the document features. In someembodiments, the first deep learning model is different from the seconddeep learning model. For example, the classifier subsystem 110 may use aNaïve Bayes classifier to assign a first document type (or “category”)to a document (where the first document type category is associated witha corresponding set of document features) and the data extractionsubsystem 112 may then use the corresponding set of document features inconjunction with the text of the document to determine a set of documentfeature values using a multi-channel transformer model, such as a dualBERT model or a Siamese BERT model. In some embodiments, the set ofdocument feature values are stored in the database(s) 132 or some otherdatastore.

FIG. 2 is a diagram that includes a logic-and-physical-architecturesystem 200 in accordance with one or more embodiments. Thelogic-and-physical-architecture system 200 may be employed, for example,to provide a database language query based on an obtained naturallanguage question. In the illustrated embodiment, the architecturesystem 200 includes document(s) 202, a document classifier 210, a firstdeep learning model 212, a document feature retriever (or “fieldretriever”) 220, a document field datastore 222, a “field” interface230, a second deep learning model (or “data extraction deep learningmodel”) 240, a model parameter repository (or “data extraction deeplearning model parameter datastore”) 242, an “extraction” interface 244.In some embodiments, the system 200 is employed to classify (or“categorize”) documents, to determine document features based on theclassifications, and to extract corresponding feature values from thedocuments. In some embodiments, the document classifier 210 retrievesvalues from a deep learning model 212, where the deep learning model 212is used for document classification. Each of the documents from the setof documents 202 may be classified and then provided to the documentfeature retriever 220. After the document classifier 210 is trained,some embodiments may use the document classifier 210 to assign one ormore document type categories to a document.

The document feature retriever 220 may retrieve fields of the documentbased on the classification performed by the document classifier 210.For example, some embodiments may classify a first document using thedocument classifier 210 to determine that the first document has acategory of “contract.” The document feature retriever 220 may determinefields based on data retrieved from a field datastore 222. The fielddatastore 222 may include records corresponding with different types ofdocuments. For example, the field datastore 222 may include a firstrecord having the title “shipment contracts—1.1” and a second recordhaving the title “shipment contracts—appendix.” The records of the fielddatastore 222 may have different record properties. For example, therecord having the title “shipment contracts—1.1” may have an associatedset of record properties such as “ship type,” “ship color,” and“container type,” and the record having the title “shipmentcontracts—backup” may have an associated set of record properties suchas “cost,” “insured amount,” and “tax.”

In some embodiments, the document feature retriever 220 includes orotherwise access a deep learning model. Parameters of the deep learningmodel of the document feature retriever 220 may be used to dynamicallydetermine document feature values based on the document typeclassifications determined using the document classifier 210. Forexample, some embodiments may provide a set of documents to a deeplearning model of the field retriever 220 as inputs for a trainingoperation, where formatting, spacing, changes in font, changes in color,or other visual indicators of documents may be detected and used asparts of the input for the training operation to determine one or morefields of a document.

Alternatively, or in addition, some embodiments may permit an interface230 to edit the values of the retrieved fields obtained using thedocument feature retriever 220. For example, the interface 230 mayemploy a field editor 232 (e.g., graphic user interface) that permits auser to modify a set of retrieved fields. This may include, for example,editing retrieved fields, adding fields to the set of retrieved fields,or removing fields from the set of retrieved fields. After permitting auser to update may update one or more records in the document featuredatastore 222 to reflect the modifications to the retrieved fields. Forexample, the field editor 232 may permit a user to use the interface 232to select to modify the name of a field titled “blue” and modify thefield by changing the name of a field titled “blue” to the string“Blues.” In response, the field retriever 220 may update a record of thedocument feature datastore 222 to replace the value “blue” with “Blues.”As described, the modifications of the retrieved fields may includeadding a field and providing a corresponding model for the field (e.g.,a field micro-artificial-intelligence-model (or “micro-ai-model”) 236)or providing field training data (e.g., field training data 234) thatcan be used to generate or train a model (e.g., a micro-ai-model)associated with the newly added field. The modifications of theretrieved fields may include deleting fields. Examples of modificationsto retrieved fields and the use of micro-ai-models are described hereinwith regard to at least FIGS. 5 and 6 . Furthermore, during use of thefield retriever 220, some embodiments may determine one or more fieldsof an input document, such as a document uploaded by a user, obtainedfrom another server, obtained from the Internet, or the like.

After retrieving the set of fields using the document feature retriever220, the extracted data may be processed by a second deep learning model240. Some embodiments may train the second deep learning model 240 basedon corpora and a plurality of document type categories associated withthe corpora. For example, some embodiments may obtain a first document,a document type category, and a set of features for a document duringtraining of the second deep learning model 240. In some embodiments, thedocument type categories used during a training operation for the seconddeep learning model 240 may be determined using the document classifier210, and where the set of features may be determined using the seconddeep learning model 240. In some embodiments, a deep learning model(e.g., a micro-ai-model) is used to obtain vales for a specific feature(or “field”). For example, a first micro-ai-model 236 may be trained toextract values for the field “date” and a second micro-ai-model 236 maybe trained to extract values for the field “days”. In such anembodiments, where the document is classified as a category associatedwith the fields “date” and “days,” the first-micro model 236 may beretrieved and employed by the data extraction model 240 to extractvalues of “date” (e.g., “1 April 2022”) from the text of the document,and the second-micro model 236 may be retrieved and employed by the dataextraction model 240 to extract values of “day” (e.g., “5:00:00 pm”)from the text of the document. As described, the extraction process mayinvolve identifying sections most relevant to a given field, andapplying a micro-ai-model for the given field to those most relevantsection. This may include, for example, identifying ten sentences withrelatively high scores for the field “date,” and applying themicro-ai-model for the word “date” to those ten sentences to extractdate values from those ten sentences. Examples of use of micro-ai-modelsto extract field values are described with regard to at least FIG. 6 .

Some embodiments may populate model parameters of the second deeplearning model 240 with values from the field retriever 220. Someembodiments may then store the extracted AI model parameters into theextracted model parameter repository 242. Some embodiments may permit auser or third-party system to use an interface 244 to modify values ofthe extracted model parameter repository 242. Alternatively, or inaddition, some embodiments may store relationships between differentdeep learning models to indicate scores associated with differentcombinations of hierarchical arrangements of different layers of deeplearning models. Processed data 260, such as document categories,features and feature values extracted, may be stored in a documentfeatures data 262. The processed data 260 may, for example, includedocuments generated based on the document categories, features andfeature values extracted.

Example Training Operations

In some embodiments, a natural language processing (NLP) computersystem, such as system 100, conducts operations for training a deeplearning model, such as data extraction deep learning model 240, fornatural language document information extraction.

In some embodiments, the deep model training operations includeobtaining, by the NLP computer system, natural language documentinformation including: (a) a category of a natural language document(e.g., a natural language document 202); (b) features (e.g., fields)associated with the natural language document; and (c) text of thenatural language document (including values of the features associatedwith the natural language document). The category of the naturallanguage document may, for example, be specified by a user or determinedusing a document classifier (e.g., as described herein with regard todetermining categories of a document using a classifier subsystem 110).The features associated with the natural language document may, forexample, be specified by a user or determined using a document fieldretriever (e.g., as described herein with regard to determining fieldsof a document using a document field retriever 220). The text of thenatural language document may, for example, be provide by a user oranother entity, or obtained from a database). In some embodiments, thenatural language document information is provided by way of the fieldeditor 232 for use in training “new” micro-ai-models for associatedfields (or “terms”). For example, if the field retriever 230 presentsthe user with a list of extracted fields that include “date” and “time,”but the user would like to include the field “company name,” then theuser may submit, by way of the field editor a request to add the field“company name” and provide corresponding field training data 234, suchas training documents including samples of text that include values ofcompany names, pre-identified values of company names, and the like.And, the field training data 234 may be employed in a trainingoperation, such as those describe here, to determine a micro-ai-modelfor the field “company name.”

In some embodiments, the deep model training operations includedetermining, by the NLP computer system using a transfer model, atransformer output including a set of hidden-state values, n-gramscorresponding to the sequences of textual elements. Determining atransformer output including a set of hidden-state values, n-gramscorresponding to the sequences of textual elements. This may include,for example, a directional encoder, such as a BERT model or dual BERTmodel, providing document feature values (e.g., field values) anddocument descriptions to first and second transformers, respectively, togenerate first and second sets of hidden values, respectively, asdescribed herein.

In some embodiments, the deep model training operations includeconducting, by the NLP computer system, average pooling on the set ofhidden-state values to generate a pooled output including a set ofaverage-pooled hidden values. The average pooling on the set ofhidden-state values to generate a pooled output including a set ofaverage-pooled hidden values may include, for example, averagingperformed on different sections of the set of hidden-state values togenerate a pooled output including average-pooled sets of hidden valuesfor the different sections. In some instances, first and second sets ofhidden values of a dual BERT may be reduced using average pooling togenerate first and second average-pooled sets of hidden values,respectively, for the first and second sets of hidden values,respectively.

In some embodiments, the deep model training operations includeconducting, by the NLP computer system, concatenating of theaverage-pooled hidden values of the pooled output to generate aconcatenated output including a set of hidden state values. Theconcatenating may include, for example, combining different hidden valuestates together (e.g., by way of average pooling and concatenationoperations) into a single set of hidden state values.

In some embodiments, the deep model training operations includedetermining, by the NLP computer system based on the concatenatedoutput, scores for sequences of elements of the natural languagedocument for different features. Determining scores for sequences ofelements of the natural language document for different features mayinclude, for example, using fully-connected neural network layers toscore n-grams such as phrases, sentences, or the like of the naturallanguage document, where the score is indicative of the n-gramscontaining a feature value.

In some embodiments, the deep model training operations includeselecting, by the NLP computer system based on the scores, a sequence ofelements associated with a feature. Selecting a sequence of elementsassociated with a feature may include, for example, selecting n-gramssuch as phrases, sentences, or the like of the natural languagedocument, associated with a score that satisfies a threshold featurevalue score.

In some embodiments, the deep model training operations includeextracting, by the NLP computer system from the sequence of elementsassociated with the feature, a value of the feature. Extracting a valueof the feature may include, for example, extracting a value of a feature(e.g., the field value of “24 April 2021”) using a deep learning modelof the field retriever 220 as described here).

Example Information Extraction Operations

In some embodiments, a natural language processing (NLP) computersystem, such as system 100, conducts operations for extractinginformation from a natural language document, such as an unstructurednatural language document 202.

In some embodiments, the information extraction operations includeobtaining, by the NLP computer system, an unstructured natural languagedocument including text sections including sequences of textualelements. The textual elements may include, for example, words, phrases,or symbols.

In some embodiments, the information extraction operations includedetermining, by the NLP computer system, n-grams corresponding to thesequences of textual elements. In some embodiments, each n-gram includesa contiguous sequence of n items from a given sample of textual elementsin the document. The n-grams may, for example, include blocks ofrelatively large textual units, such as words or relatively smalltextual units, such as syllables. As an example, in a document includingthe phrase “The contract effective date is 1 Mar. 2022”, the phrase maybe segmented into the following six 3-grams: “The contract effective”,“contract effective date”, “effective date is”, “date is 1”, “is 1March” and “1 Mar. 2022”).

In some embodiments, the information extraction operations includedetermining, by the NLP computer system, a count of the n-grams for thedocument. The count of the n-grams may be, for example, for each ofdifferent n-grams, a count of the number of times the n-gram appears inthe sequences of textual elements of the document. Continuing with theprior example, if the n-gram “contract effective date” appears ninetimes in the document, the n-gram “contract effective date” may have acount of “9” for the document.

In some embodiments, the information extraction operations includedetermining, by the NLP computer system and based on the count ofn-grams, a vector of the unstructured natural language document. Thevector of the unstructured natural language document may, for example,be determined using term frequency-inverse document frequency (TF-IDF)scoring. In a vector representing the unstructured natural languagedocument, each dimension of the vector may correspond to a separateterm. The definition of term depends on the application. Terms may bedefined, for example, as single words, keywords, or longer phrases. Ifwords are chosen to be the terms, the dimensionality of the vector isthe number of words in the vocabulary (the number of distinct wordsoccurring in the corpus). If, for example, a term occurs in thedocument, its value in the vector is non-zero, and generally increaseswith the number of occurrences in the document. These vector values,also known as (term) weights, may be determined using, for example,TF-IDF weighting (or “scoring”). A TF-IDF weight (or “score”) for a termmay be a numerical statistic that is intended to reflect how important aword is to a document in a collection or corpus. A TF-IDF score for aterm may, for example, be a product of a term frequency (TF) of the termwithin a document and an inverse document frequency (IDF) (e.g., TF-IDFscore=TF*IDF). The term frequency (TF) of a term may be defined by therelative frequency of the term within document. The inverse documentfrequency (IDF) may be a measure of how much information the wordprovides, e.g., if it is common or rare across all documents in acollection or corpus. The inverse document frequency (IDF) may bedefined as the logarithmically scaled inverse fraction of the documentsthat contain the word (obtained by dividing the total number ofdocuments by the number of documents containing the term, and thentaking the logarithm of that quotient).

In some embodiments, the information extraction operations includedetermining, by the NLP computer system and based on the vector using afirst deep learning model (such as model 212), a category for theunstructured natural language document. The category for theunstructured natural language document may be, for example, a “realestate contract” category determined using the first deep learning modeland a vector for the document. In such an embodiment, the first deeplearning model may be used to determine that the vector for the documentcorresponds to the “real estate contract” category. The first deeplearning model may be, for example, a Naïve Bayes classifier, a randomforest classifier, or a logistic regression classifier.

In some embodiments, the information extraction operations includeobtaining, by the NLP computer system in response to determining thecategory and from a database storing sets of document featuresassociated with document categories, a set of document features (e.g.,feature of document field datastore 222) associated with the category.The set of document features may include, for example, a predeterminedset of features that are associated with the category. Continuing withthe prior example of a real estate contract, the document features mayinclude, for example, fields of “purchase date,” “property address,”“purchaser,” “seller” and the like that are associated with the “realestate contract” category. The document features may include, forexample, numbers, words, symbols, hyperlinks, images, other media, orthe like. The document features for a category may be determined using ause a deep learning model, such as Siamese BERT model or triplet networkmodel as described herein.

In some embodiments, the information extraction operations includedetermining, by the NLP computer system in response to determining theset of document features and using a second deep learning model (e.g.,data extraction deep learning model 240), a score for a text section ofthe unstructured natural language document based on the set of featuresand text of the unstructured natural language document. The second deeplearning model may be, for example, a multi-channel transformer model,such as a Siamese BERT or triple network model. In such an embodiment,determining the score may include (a) providing, to a first channel ofthe multi-channel transformer, the set of document features, andproviding, to a second channel of the multi-channel transformer, textualelements of the unstructured natural language document, where themulti-channel transformer determines the score for text sections of thedocument based on the first channel processing the set of documentfeatures and the second channel processing the textual elements. Forexample, first a document may be passed to an NLP system (e.g., aclassifier) that is employed to determine the category (or “type”) ofthe document. A second stage classifier may be employed to determinedifferent sections of the document where relevant information may bepresent and to order these sections in the order of confidence scores.Continuing with the prior example of a real estate contract, where thetext sections are broken into sentences within the document, the seconddeep learning model may calculate scores for each sentence text sectionof the document, including respective scores for sentences defining“purchase date,” “property address,” “purchaser,” “seller”, and soforth. In some embodiments, a micro-ai-model may be employed forextracting values for the features (or “fields”). For example, where thedocument is categorized as a real estate contract, and the fields of“purchase date,” “property address,” “purchaser,” and “seller” areidentified as fields for the real estate contract document, a separatemicro-ai-model 236 for each of “purchase date,” “property address,”“purchaser,” and “seller” may be applied to the text of the document toextract values for the respective fields.

In some embodiments, the information extraction operations includeselecting, by the NLP computer system based on the score, the textsection. In some embodiments, text sections having scores that satisfy athreshold are assessed for extraction of values of document feature.Continuing with the prior example of a real estate contract, where thesentences defining “purchase date,” “property address,” “purchaser,”“seller”, and so forth are assigned text section scores that satisfy(e.g., are above) a priority score threshold, the sentences may beprovided to a finer-grain deep learning model (e.g., a respectivemicro-ai-model for a given feature) to determine a document featurevalue associated with a document feature. An example of a text sectionselection operation (e.g., using micro-ai-models) is described withregard to FIG. 6 . The finer-grain deep learning model may be, forexample, a transformer model, such as a Siamese BERT model.

In some embodiments, the information extraction operations includedetermining, by the NLP computer system based on the text section, afeature value. Continuing with the prior example of a real estatecontract, this may include the finer-grain deep learning model assessingthe sentences defining “purchase date,” “property address,” “purchaser,”“seller”, and so forth to determine respective values for “purchasedate” (e.g., “1 April 2022”), “property address” (e.g., “101 MainStreet”), “purchaser” (e.g., “John Doe”), “seller” (e.g., “Jane Smith”)and so forth).

In some embodiments, the information extraction operations includestoring, by the NLP computer system in a database (e.g., documentfeatures database 262), the feature value(s) in association with theunstructured natural language document. Continuing with the priorexample of a real estate contract, the respective values for the fieldsof “purchase date” (e.g., “1 April 2022”), “property address” (e.g.,“101 Main Street”), “purchaser” (e.g., “John Doe”), “seller” (e.g.,“Jane Smith”) and so forth, may be stored in a database in associationwith the real estate contract.

In some embodiments, the textual elements are words, the unstructurednatural language document is a contract document outlining contractterms, and the feature values include numerical values for one or moreof the contract terms. In such an embodiment, a second contract may begenerated based on the numerical values for one or more of the contractterms. The second contract may be, for example, a self-executingcontract including computer code defining the contract terms andconfigured to be executed to enforce the contract terms. In such anembodiment, generating the second contract may include generating (e.g.,by the NLP computer system) the computer code defining the contractterms and storing, on a distributed ledger peer-to peer-network (e.g., ablockchain network). The computer code may include conditionalstatements defining the contract terms, and the computer code stored onthe distributed ledger peer-to peer-network may be configured to beexecuted to enforce the conditional statements of the contract terms.For example, the contract may be a real-estate contract that specifiesthat a financial transaction (e.g., a monetary purchase payment from anescrow account of a buyer) be accomplished in response to the occurrenceof a predefined event (e.g., in response to closing of a real estatetransaction for the sale of a real estate property by a seller), and thecomputer code stored on the distributed ledger peer-to peer-network maybe executed in response to the occurrence of a predefined event toaccomplish the financial transaction (e.g., the computer code maymonitor for the closing of the real estate transaction and, upondetermining that the real estate transaction has closed, transfer moniesfor the purchase payment from the escrow account, into the sellersaccount).

Example Flowcharts

The processes presented in this disclosure are intended to beillustrative and non-limiting. In some embodiments, for example, themethods may be accomplished with one or more additional operations notdescribed or without one or more of the operations discussed.Additionally, the order in which the processing operations of themethods are illustrated (and described below) is not intended to belimiting. In some embodiments, the methods may be implemented in one ormore processing devices (e.g., a digital processor, an analog processor,a digital circuit designed to process information, an analog circuitdesigned to process information, a state machine, or other mechanismsfor electronically processing information). The processing devices mayinclude one or more devices executing some or all of the operations ofthe methods in response to instructions stored electronically on anelectronic storage medium. The processing devices may include one ormore devices configured through hardware, firmware, or software to bespecifically designed for the execution of one or more of the operationsof the methods.

In some embodiments, the computing environment 100 may execute one ormore process routines described in this disclosure. In some embodiments,the various operations of the processes 300 or 400 may be executed in adifferent order, operations may be omitted, operations may bereplicated, additional operations may be included, some operations maybe performed concurrently, some operations may be performedsequentially, and multiple instances of the processes 300 or 400 may beexecuted concurrently, none of which is to suggest that any otherdescription herein is limited to the arrangement described. In someembodiments, the operations of the processes 300 and 400 may beeffectuated by executing program code stored in one or more instances ofa machine-readable non-transitory medium, which in some cases mayinclude storing different subsets of the instructions on differentphysical embodiments of the medium and executing those different subsetswith different processors, an arrangement that is consistent with theuse of the singular term “medium” herein.

FIG. 3 is a flowchart diagram that illustrates a method (or “process”)300 for training a deep learning model for information extraction inaccordance with one or more embodiments of the present technique. Insome embodiments, operations of the process 300 include obtaining a setof input text, an associated set of document type categories, or anassociated set of document features, as indicated for block 304. The setof input text may be obtained from an upload performed by a user using aclient computing device. In some embodiments, the associated set ofdocument type categories are determined using a document classifierdescribed elsewhere in this disclosure. Alternatively, or in addition, acategory of the set of document type categories may be provided by theuser during the upload or during a later update to metadata associatedwith the document. Furthermore, some embodiments may include obtainingan associated set of document features using a field retriever, wherefields of the field retriever may be obtained from a document featuresdatastore, as described elsewhere in this disclosure.

In some embodiments, operations of the process 300 include determining aset of one or more outputs based on the set of input text using a set oftransformers, as indicated for block 308. As described elsewhere in thisdisclosure, some embodiments may use a transformer model to extractvalues from a document, where the transformer model may include aplurality of layers or may include multiple sets of layers. For example,some embodiments may use a directional encoders model such as a singleBERT structure to determine a transformer output. Some embodiments mayperform one or more preprocessing operations to the set of input text,such as lemmatizing, stemming, rooting, or the like. Some embodimentsmay convert in-vocabulary words of the input text into a list of tokensand split out-of-vocabulary words into a set of sub-words that are partof the vocabulary. Using the single BERT structure, some embodiments maygenerate three sets of values, such as a first matrix of values, asecond matrix of values, and a third matrix of values, where the firstmatrix of values may represent a matrix of token values, the secondmatrix of values may represent a mask matrix, and the third matrix ofvalues may represent sequence identifiers. Some embodiments may thenprovide the three sets of values to a transformer to obtain a finalhidden-state.

Alternatively, or in addition, some embodiments may use a dual BERTmodel, where using a dual BERT model includes using two differenttransformers having a same composition but being trained with differentinputs. For example, some embodiments may use a dual BERT structure byproviding a first transformer with document feature values of a set oftraining documents and providing a second transformer with acorresponding set of document descriptions of the set of trainingdocuments. After being provided with the document feature values, thefirst transformer may provide a first tuple of three sets of values.Similarly, after being provided with the document descriptions, thesecond transformer may provide a second tuple of three sets of values.Some embodiments may then determine a corresponding first set of hiddenvalues using the first transformer and a second set of hidden valuesusing the second transformer.

Some embodiments may train a Siamese BERT architecture to extract datafrom a document. Using a Siamese BERT architecture may includeconcurrently providing document feature values and a corresponding setof document descriptions of the set of training documents to a SiameseBERT transformer. By using the Siamese BERT transformer, the same set oftransformer model parameters may be used to determine a first set ofhidden values and a second set of hidden values.

In some embodiments, operations of the process 300 include performingaverage pooling on the set of transformer outputs, as indicated forblock 312. Some embodiments may perform average pooling by determiningaverages along different sections of the hidden states and using theaverages as a set of average pooled outputs. In some embodiments, eachset of the first and second set of hidden values of a dual BERT modelmay be reduced using average pooling into a corresponding firstaverage-pooled set of hidden values and a corresponding secondaverage-pooled set of hidden values.

In some embodiments, operations of the process 300 include concatenatingthe average pooling outputs, as indicated for block 316. Concatenatingthe average-pooled outputs may include combining different hidden statevalues together into a single set of hidden state values.

In some embodiments, operations of the process 300 include determining aset of scores based on the concatenated set of average-pooled outputs,as indicated for block 320. In some embodiments, such as for a dual BERTor Siamese BERT model, some embodiments may pass a concatenated set ofhidden state values to a set of fully connected neural network layers.The output of the fully connected neural network layers may be a scoreassigned to a phrase, sentence, or another set of n-grams indicating apriority score of relevance. Some embodiments may then select one ormore of the sets of n-grams based on the score to determine which set ofn-grams to use to extract relevant information for a field. For example,some embodiments may assign a priority score of “0.99” to the sentence“Effective date->the contract will be valid from 24 April 2021.” Someembodiments may then obtain the date “24 April 2021” from the sentence.

FIG. 4 is a flowchart diagram that illustrates a method (or “process”)400 of extracting information of a document in accordance with one ormore embodiments of the present technique FIG. 4 is a flowchart diagramthat illustrates a method of extracting information of a document inaccordance with one or more embodiments of the present technique. Insome embodiments, operations of the process 400 include obtaining a setof documents, as indicated by block 404. Obtaining the set of documentsmay include receiving corpora from a datastore, a user interface, anapplication program interface, or the like. For example, someembodiments may obtain the set of documents from a user via an uploadfrom a user computing device. Various types of documents may beprovided, such as plain text documents, PDF documents, Microsoft Word™documents, or the like.

In some embodiments, operations of the process 400 include determining aset of document type categories based on the set of documents, asindicated for block 408. During the categorization of the set ofdocuments, some embodiments may perform one or more document analysis ortransformation operations to quantify different features of documents.For example, some embodiments may perform operations such as countvectorization (i.e., one-hot encoding), TF-IDF vectorization, or thelike. For example, some embodiments may perform a set of TF-IDFoperations to determine the relative importance of words of a documentbased on a plurality of documents. Some embodiments may then providefeatures obtained from these operations to one or more classifiers toclassify a document with one or more document type categories. Varioustypes of classifiers may be used, such as a random forest classifier,logistic regression classifier, naïve Bayes classifier, or the like. Forexample, some embodiments may provide a set of n-grams such as words,phrases, or symbols and their associated TF-IDF scores to a naïve Bayesclassifier to determine that the document associated with the set ofn-grams should be assigned the document type category “real estatecontract.”

In some embodiments, operations of the process 400 include obtaining aset of document feature associated with the set of document typecategories, as indicated for block 412. Some embodiments may directlyobtain a set of document features based on a document type category.Some embodiments may retrieve one or more rules from a rules engine oruse other low-code or no-code engines to determine one or more featuresassociated with a document. Some embodiments may retrieve the rules,values of the rules, or other information based on the set of documenttype categories assigned to the document using a document classifierdescribed in this disclosure.

In some embodiments, operations of the process 400 include obtaining aset of document feature values associated with the set of documentfeatures, as indicated for block 416. Some embodiments may use a deeplearning model to obtain the set of document feature values, where theset of document feature values may include numbers, words, symbols,hyperlinks, images, other media, or the like. In some embodiments, thedeep learning model used to determine the set of document feature valuesmay include a multi-channel transformer model, such as a Siamese BERTmodel or triplet network model. For example, some embodiments may use aSiamese BERT model to obtain a set of document feature values, where afirst input for a first channel of the Siamese BERT model may includethe set of document features associated with a set of document typeclassifiers, and where a second input for a second channel of theSiamese BERT model may include a set of document text.

Some embodiments may use a hierarchical set of deep learning models toextract document feature values. For example, some embodiments may use afirst dual BERT or Siamese BERT model to assign priority scores to eachsentence (or other text section) of a document with respect to a set ofdocument features. Some embodiments may then provide each sentence thatis assigned a priority score greater than a priority score threshold toa finer-grain deep learning model to determine a document feature valueassociated with a document feature. The finer-grain deep learning modelmay be another transformer model, such as another Siamese BERT model, ormay be a different neural network model. In some embodiments, thefiner-grain deep learning model may use one or more context parametersdetermined using other operations described in this disclosure, such asa priority score, a TF-IDF result, a category, or the like. For example,in an instance involving the following three sentences: (1) “The startdate is 24^(th) April 2019”, (2) “The end date is 24^(th) April 2022”,and (3) “This document is signed by Mr. John Peter, if the system isattempting to identify a start date, all the three sentences may beranked by the classifier based on the relevance score for start date.Sentence (1) may be ranked first/highest because it is determined tocontain date information including a start date, sentence (2) may beranked second because it is determined to contain date information, andsentence (3) may be ranked third/lowest because it is determined to havelittle to no relevance to the field (e.g., it is determined to notcontain any date information). In such a scenario, sentence (1) may beprocessed first for the date extraction, based on it having the highestranking and the start date results (e.g., “24^(th) April 2019”) may beprovided as the result for the start date field.

Some embodiments may determine a plurality of values for a documentfeature. For example, a first document feature may be labeled with thestring “price.” Some embodiments may use a trained deep learning modelsuch as a trained Siamese BERT model to extract the value “30” inAssociation with the document feature “price” based on text of a firstpage of a document. Some embodiments may then use the same trainedSiamese BERT model to extract the document feature value “40” inAssociation with the same document feature “price” based on text of asecond page of a document. Some embodiments may store both instances ofthe document feature value in a record of the document. As describedelsewhere in this disclosure, some embodiments may apply one or morerules to verify the different document feature values or identify amismatch between the document feature values.

If a determination is made that the set of criteria is satisfied basedon the set of document type values, operations of the flowchart 400 mayproceed to operations described for block 424. Otherwise, operations ofthe flowchart 400 may proceed to operations described for block 430.

In some embodiments, operations of the process 400 include sending anotification or modifying a version of the unstructured natural languagedocument, as indicated for block 424. Some embodiments may perform oneor more actions based on the outcomes of whether or not the set ofcriteria are satisfied. In some embodiments, the actions may includeproviding one or more notifications to an end-user or otherwise inassociation with a document based on the criteria. Alternatively, or inaddition, some embodiments may perform an automated action such ascorrecting a quantitative mismatch or indicating that one or morecontract terms is rejected.

In some embodiments, operations of the process 400 include storing theset of obtained document feature values or other values in a recordassociated with the document, as indicated for block 430. For example,some embodiments may store the set of document feature values in a datalake. The values of the data lake may be used for various operations,such as additional training of one or more deep learning models,quantitative analysis, comparison to a set of criteria, or the like.

Example Micro-AI-Model Generation

FIG. 5 is a flowchart diagram that illustrates a method (or “process”)500 employing micro-ai-models in accordance with one or more embodimentsof the present technique. The operations of method 500 may be performed,for example, by a NLP computer system, such as system 100.

In some embodiments, operations of the process 500 include obtaining adocument, as indicated for block 502. This may include obtaining anunstructured natural language document having text sections includingsequences of textual elements. For example, this may include a NLPcomputer system obtaining a document 202, as described here with regardto the document classifier 210. The document may be, for example, areal-estate contract having fields of “date” and “time” embedded in thetext thereof.

In some embodiments, operations of the process 500 include determining adocument category, as indicated for block 504. This may includeconducting a classification operation to determine a category (or“classification”) of the unstructured natural language document.Continuing with the prior example, this may include the NLP computersystem (e.g., document classifier 210) conducting a classificationoperation (e.g., using first deep learning model 212) to determine thatthe document 202 is a real-estate contract.

In some embodiments, operations of the process 500 include determiningdocument fields for a category for a document, as indicated for block506. This may include conducting a field retrieving operation todetermine one or more features (or “fields”) associated with thecategory of the unstructured natural language document. Continuing withthe prior example, this may include the NLP computer system (e.g., fieldretriever 220) conducting a field retrieving operation to determine thatthe fields “date” and “time” are associated with the real-estatecontract document category.

In some embodiments, operations of the process 500 include presentingdocument fields for a document, as indicated for block 508. This mayinclude presenting a listing of the features (or “fields”) associatedwith the category of the unstructured natural language document.Continuing with the prior example, this may include the NLP computersystem (e.g., field editor 232 of the interface 230 of the fieldretriever 220) displaying on a graphical user interface for viewing andselection by a user, a listing the fields “date” and “time” determinedto be associated with the real-estate contract document category. Thelisting of fields may be interactive, allowing a user to select tomodify a displayed field, select to add a field to the listing, orselect to delete a field from the listing.

In response to receiving a user selection to modify a field of thelisting (block 510), the process 500 may proceed to updating theassociated field micro-ai-model. For example, in response to the NLPcomputer system (e.g., field editor 232 of the interface 230 of thefield retriever 220) receiving a request to edit the field “date” to“day,” the NLP computer system (e.g., field editor 232) may update themicro-ai-model 236 (stored in datastore 242) currently associated with“date” to be associated with “day,” such that values extracted using themicro-ai-model 236 are labeled as “day” as opposed to “date.”

In response to receiving a user selection to add a field to the listing(block 512), the process 500 may proceed to obtaining training data forthe field (block 518) and conducting corresponding training (block 520)(e.g., using the obtained training data) to generate a micro-ai-modelfor the field to be added. For example, in response to the NLP computersystem (e.g., field editor 232) receiving a request to add the field“purchaser,” the NLP computer system (e.g., field editor 232) may promptthe user to provide the field name and to upload relevant field trainingdata 234 for the field. The field training data 234 for the field may,for example, include “positive samples” including document textcontaining names of purchasers, and a listing of the names of thepurchasers for use in training a “purchaser” micro model 236. Inresponse to receiving the field training data 234, the NLP computersystem (e.g., field editor 232) may conduct a training operation (block520) to generate a micro-ai-model 236 for the “purchaser” field (or“term”), and the “purchaser” micro-ai-model 236 may be stored in thedatastore 242 for use in extracting values for the “purchaser” field.

In response to receiving a user selection to remove a field of thelisting (block 514), the process 500 may proceed to removing theassociated field (block 522). For example, in response to the NLPcomputer system (e.g., field editor 232) receiving a request to removethe field “time,” the NLP computer system (e.g., field editor 232) mayremove the field from the association with real-estate contract typedocuments and delete a “time” micro-ai-model 236 (from the datastore242). Thus, the associated listing of features may not include “time”such that values for “time” will not be extracted from the real estatedocument. And, the next time a real-estate document is encountered,“time” will not be listed as an associated feature, and values for“time” will not be extracted from the real estate document.

In some embodiments, operations of the process 500 include storingmicro-ai-models for fields for the document category, as indicated forblock 522. This may include storing the added and modifiedmicro-ai-models for use in extracting values from the document.Continuing with the prior example, this may include the NLP computersystem (e.g., field editor 232) storing the modified “day”micro-ai-model and the new “purchaser” micro-ai-model in datastore 242.

Example Micro-AI-model Extraction

FIG. 6 is a diagram that illustrates a method (or “process”) 600 ofemploying micro-ai-models in accordance with one or more embodiments ofthe present technique. The operations of method 600 may be performed,for example, by a NLP computer system, such as system 100.

In some embodiments, operations of the process 600 include obtainingdocuments and document fields, as indicated for block 602. This mayinclude obtaining an unstructured natural language document having textsections including sequences of textual elements, and an identificationof features (or “fields”) associated with the document. Continuing withthe prior example, this may include the NLP computer system (dataextraction deep learning model 240) receiving (as a result ofcategorization and field retrieving operations), the real-estatedocument and a listing of the fields “day” and “purchaser.”

In some embodiments, operations of the process 600 include obtainingmicro-ai-models for document fields, as indicated for block 604. Thismay include obtaining a micro-ai-model associated with each of thefeatures (or “fields”) associated with a document. Continuing with theprior example, this may include the NLP computer system (data extractiondeep learning model 240) retrieving, from a model database 606 (e.g.,the datastore 242), the “day” micro-ai-model 236 and the “purchaser”micro-ai-model 236.

In some embodiments, operations of the process 600 include determiningn-grams for the document, as indicated for block 608, and associatedn-gram sequences, as indicated for block 610. This may includedetermining n-gram of the document and assembling them into a set ofn-gram sequences for the document. Continuing with the prior example,this may include the NLP computer system (data extraction deep learningmodel 240) determining n-gram of the document and corresponding n-gramsequences that are sentences (or similar phrases) contained in thereal-estate contract.

In some embodiments, operations of the process 600 include, for eachfield, providing to a transformer (612), the correspondingmicro-ai-model for the document field and the n-gram sequences for usein determining, for each of the fields, scores for each of the n-gramsequences. Continuing with the prior example, in a first iteration, the“day” micro-ai-model 236 and the sentences of the real-estate contractmay be provided to the transformer 612 to generate a set of sequenceoutput scores (block 618) that reflects the relevance of each sentenceto the “day” field (e.g., a score for each sentence that indicates howlikely the sentence is to contain a value for the “day” field). In asecond iteration, the “purchaser” micro-ai-model 236 and the sentencesof the real-estate contract may be provided to the transformer 612 togenerate a set of sequence output scores (block 618) that reflect therelevance of each sentence to the “purchaser” field (e.g., a score foreach sentence that indicates how likely the sentence is to contain avalue for the “purchaser” field). This may include the NLP computersystem (data extraction deep learning model 240) determining n-gramsequences that are sentences (or similar phrases) contained in thereal-estate contract. The transformer may, for example, employ multiplelayers (614) (as described here) to apply the given micro-ai-model tothe input n-gram sequences to generate the sets of sequence outputscores (block 618). The output of the layer (614) may be provided to adense layer (616) that provides the sets of sequence output scores(block 618). The dense layer (616) may be, for example, a fullyconnected linear layer of the neural networks.

In some embodiments, operations of the process 600 include selectingsequences, as indicated for block 620. This may include selecting, foreach field, n-gram sequences associated with a score that satisfies athreshold. Continuing with the prior example, this may include the NLPcomputer system (data extraction deep learning model 240) determining,from the sequence output scores for the “day” field, a given number ofthe sentences with the highest score or the sentences having a scoreabove a given minimum threshold score. A similar process may beconducted for each of the other document fields.

In some embodiments, operations of the process 600 include extractingvalues for the fields from the selected sequences, as indicated forblock 624, and storing those values, as indicated at block 624. This mayinclude, for each field, conducting an extraction operation on theselected sequences for the field to determine field values therefore,and storing those values. Continuing with the prior example, this mayinclude the NLP computer system (data extraction deep learning model240) applying, to the set of sentences selected for the “day” field, adata extraction model for the term “day” (e.g., a micro-ai-model 236 forthe field/term day) to extract values for the field “day” from thereal-estate contract. A similar process may be conducted for each of theother document fields. The extracted values (e.g., for “day,”“purchaser,” and so forth) may be stored in the database 262.

Example Computing Device

FIG. 7 is a diagram that illustrates an example computing device inaccordance with one or more embodiments of the present technique. FIG. 7is a diagram that illustrates an example computing device 1000 inaccordance with one or more embodiments of the present technique.Various portions of systems and methods described herein, may include orbe executed on one or more computer systems similar to computer system1000. Further, processes and modules described may be executed by one ormore processing systems similar to that of computer system 1000.

Computer system 1000 may include one or more processors (e.g.,processors 1010 a-1010 n) coupled to System memory 1020, an input/outputI/O device interface 1030, and a network interface 1040 via aninput/output (I/O) interface 1050. A processor may include a singleprocessor or a plurality of processors (e.g., distributed processors). Aprocessor may be any suitable processor capable of executing orotherwise performing instructions. A processor may include a centralprocessing unit (CPU) that carries out program instructions to performthe arithmetical, logical, and input/output operations of computersystem 1000. A processor may execute code (e.g., processor firmware, aprotocol stack, a database management system, an operating system, or acombination thereof) that creates an execution environment for programinstructions. A processor may include a programmable processor. Aprocessor may include general or special purpose microprocessors. Aprocessor may include one or more microcontrollers. A processor mayreceive instructions and data from a memory (e.g., System memory 1020).Computer system 1000 may be a uni-processor system including oneprocessor (e.g., processor 1010 a), or a multi-processor systemincluding any number of suitable processors (e.g., 1010 a-1010 n).Multiple processors may be employed to provide for parallel orsequential execution of one or more portions of the techniques describedherein. Processes, such as logic flows, described herein may beperformed by one or more programmable processors executing one or morecomputer programs to perform functions by operating on input data andgenerating corresponding output. Processes described herein may beperformed by, and apparatus can also be implemented as, special purposelogic circuitry, e.g., an FPGA (field programmable gate array) or anASIC (application specific integrated circuit). Computer system 1000 mayinclude a plurality of computing devices (e.g., distributed computersystems) to implement various processing functions.

I/O device interface 1030 may provide an interface for connection of oneor more I/O devices 1060 to computer system 1000. I/O devices mayinclude devices that receive input (e.g., from a user) or outputinformation (e.g., to a user). I/O devices 1060 may include, forexample, graphical user interface presented on displays (e.g., a cathoderay tube (CRT) or liquid crystal display (LCD) monitor), pointingdevices (e.g., a computer mouse or trackball), keyboards, keypads,touchpads, scanning devices, voice recognition devices, gesturerecognition devices, printers, audio speakers, microphones, cameras, orthe like. I/O devices 1060 may be connected to computer system 1000through a wired or wireless connection. I/O devices 1060 may beconnected to computer system 1000 from a remote location. I/O devices1060 located on remote computer system, for example, may be connected tocomputer system 1000 via a network and network interface 1040.

Network interface 1040 may include a network adapter that provides forconnection of computer system 1000 to a network. Network interface 1040may facilitate data exchange between computer system 1000 and otherdevices connected to the network. Network interface 1040 may supportwired or wireless communication. The network may include an electroniccommunication network, such as the Internet, a local area network (LAN),a wide area network (WAN), a cellular communications network, or thelike.

System memory 1020 may be configured to store program instructions 1024or data 1011. Program instructions 1024 may be executable by a processor(e.g., one or more of processors 1010 a-1010 n) to implement one or moreembodiments of the present techniques. Program instructions 1024 mayinclude modules of computer program instructions for implementing one ormore techniques described herein with regard to various processingmodules. Program instructions may include a computer program (which incertain forms is known as a program, software, software application,script, or code). A computer program may be written in a programminglanguage, including compiled or interpreted languages, or declarative orprocedural languages. A computer program may include a unit suitable foruse in a computing environment, including as a stand-alone program, amodule, a component, or a subroutine. A computer program may or may notcorrespond to a file in a file system. A program may be stored in aportion of a file that holds other programs or data (e.g., one or morescripts stored in a markup language document), in a single filededicated to the program in question, or in multiple coordinated files(e.g., files that store one or more modules, sub programs, or portionsof code). A computer program may be deployed to be executed on one ormore computer processors located locally at one site or distributedacross multiple remote sites and interconnected by a communicationnetwork.

System memory 1020 may include a tangible program carrier having programinstructions stored thereon. A tangible program carrier may include anon-transitory, computer-readable storage medium. A non-transitory,computer-readable storage medium may include a machine readable storagedevice, a machine readable storage substrate, a memory device, or anycombination thereof. Non-transitory, computer-readable storage mediummay include non-volatile memory (e.g., flash memory, ROM, PROM, EPROM,EEPROM memory), volatile memory (e.g., random access memory (RAM),static random access memory (SRAM), synchronous dynamic RAM (SDRAM)),bulk storage memory (e.g., CD-ROM and/or DVD-ROM, hard-drives), or thelike. System memory 1020 may include a non-transitory, computer-readablestorage medium that may have program instructions stored thereon thatare executable by a computer processor (e.g., one or more of processors1010 a-1010 n) to cause the subject matter and the functional operationsdescribed herein. A memory (e.g., System memory 1020) may include asingle memory device and/or a plurality of memory devices (e.g.,distributed memory devices). Instructions or other program code toprovide the functionality described herein (such as the functions of thesystem 100) may be stored on a tangible, non-transitory,computer-readable media. The instructions may be executable by acomputer processor to cause the associated operations to be performed.In some cases, the entire set of instructions may be stored concurrentlyon the media, or in some cases, different parts of the instructions maybe stored on the same media at different times.

I/O interface 1050 may be configured to coordinate I/O traffic betweenprocessors 1010 a-1010 n, System memory 1020, network interface 1040,I/O devices 1060, and/or other peripheral devices. I/O interface 1050may perform protocol, timing, or other data transformations to convertdata signals from one component (e.g., System memory 1020) into a formatsuitable for use by another component (e.g., processors 1010 a-1010 n).I/O interface 1050 may include support for devices attached throughvarious types of peripheral buses, such as a variant of the PeripheralComponent Interconnect (PCI) bus standard or the Universal Serial Bus(USB) standard.

Embodiments of the techniques described herein may be implemented usinga single instance of computer system 1000 or multiple computer systems1000 configured to host different portions or instances of embodiments.Multiple computer systems 1000 may provide for parallel or sequentialprocessing/execution of one or more portions of the techniques describedherein.

Those skilled in the art will appreciate that computer system 1000 ismerely illustrative and is not intended to limit the scope of thetechniques described herein. Computer system 1000 may include anycombination of devices or software that may perform or otherwise providefor the performance of the techniques described herein. For example,computer system 1000 may include or be a combination of acloud-computing system, a data center, a server rack, a server, avirtual server, a desktop computer, a laptop computer, a tabletcomputer, a server device, a client device, a mobile telephone, apersonal digital assistant (PDA), a mobile audio or video player, a gameconsole, a vehicle-mounted computer, or a GPS device, or the like.Computer system 1000 may also be connected to other devices that are notillustrated, or may operate as a stand-alone system. In addition, thefunctionality provided by the illustrated components may in someembodiments be combined in fewer components or distributed in additionalcomponents. Similarly, in some embodiments, the functionality of some ofthe illustrated components may not be provided or other additionalfunctionality may be available.

Those skilled in the art will also appreciate that while various itemsare illustrated as being stored in memory or on storage while beingused, these items or portions of them may be transferred between memoryand other storage devices for purposes of memory management and dataintegrity. Alternatively, in other embodiments some or all of thesoftware components may execute in memory on another device andcommunicate with the illustrated computer system via inter-computercommunication. Some or all of the system components or data structuresmay also be stored (e.g., as instructions or structured data) on acomputer-accessible medium or a portable article to be read by anappropriate drive, various examples of which are described above. Insome embodiments, instructions stored on a computer-accessible mediumseparate from computer system 1000 may be transmitted to computer system1000 via transmission media or signals such as electrical,electromagnetic, or digital signals, conveyed via a communication mediumsuch as a network or a wireless link. Various embodiments may furtherinclude receiving, sending, or storing instructions or data implementedin accordance with the foregoing description upon a computer-accessiblemedium. Accordingly, the present techniques may be practiced with othercomputer system configurations.

In block diagrams, illustrated components are depicted as discretefunctional blocks, but embodiments are not limited to systems in whichthe functionality described herein is organized as illustrated. Thefunctionality provided by each of the components may be provided bysoftware or hardware modules that are differently organized than ispresently depicted, for example such software or hardware may beintermingled, conjoined, replicated, broken up, distributed (e.g. withina data center or geographically), or otherwise differently organized.The functionality described herein may be provided by one or moreprocessors of one or more computers executing code stored on a tangible,non-transitory, machine readable medium. In some cases, notwithstandinguse of the singular term “medium,” the instructions may be distributedon different storage devices associated with different computingdevices, for instance, with each computing device having a differentsubset of the instructions, an implementation consistent with usage ofthe singular term “medium” herein. In some cases, third party contentdelivery networks may host some or all of the information conveyed overnetworks, in which case, to the extent information (e.g., content) issaid to be supplied or otherwise provided, the information may beprovided by sending instructions to retrieve that information from acontent delivery network.

The reader should appreciate that the present application describesseveral independently useful techniques. Rather than separating thosetechniques into multiple isolated patent applications, applicants havegrouped these techniques into a single document because their relatedsubject matter lends itself to economies in the application process. Butthe distinct advantages and aspects of such techniques should not beconflated. In some cases, embodiments address all of the deficienciesnoted herein, but it should be understood that the techniques areindependently useful, and some embodiments address only a subset of suchproblems or offer other, unmentioned benefits that will be apparent tothose of skill in the art reviewing the present disclosure. Due to costsconstraints, some techniques disclosed herein may not be presentlyclaimed and may be claimed in later filings, such as continuationapplications or by amending the present claims. Similarly, due to spaceconstraints, neither the Abstract nor the Summary of the Inventionsections of the present document should be taken as containing acomprehensive listing of all such techniques or all aspects of suchtechniques.

It should be understood that the description and the drawings are notintended to limit the present techniques to the particular formdisclosed, but to the contrary, the intention is to cover allmodifications, equivalents, and alternatives falling within the spiritand scope of the present techniques as defined by the appended claims.Further modifications and alternative embodiments of various aspects ofthe techniques will be apparent to those skilled in the art in view ofthis description. Accordingly, this description and the drawings are tobe construed as illustrative only and are for the purpose of teachingthose skilled in the art the general manner of carrying out the presenttechniques. It is to be understood that the forms of the presenttechniques shown and described herein are to be taken as examples ofembodiments. Elements and materials may be substituted for thoseillustrated and described herein, parts and processes may be reversed oromitted, and certain features of the present techniques may be utilizedindependently, all as would be apparent to one skilled in the art afterhaving the benefit of this description of the present techniques.Changes may be made in the elements described herein without departingfrom the spirit and scope of the present techniques as described in thefollowing claims. Headings used herein are for organizational purposesonly and are not meant to be used to limit the scope of the description.

As used throughout this application, the word “may” is used in apermissive sense (i.e., meaning having the potential to), rather thanthe mandatory sense (i.e., meaning must). The word “set” when used as anoun include a single item or a plurality of items, such that the phrase“set of items” may refer to either a single item or multiple items. Thewords “include”, “including”, and “includes” and the like meanincluding, but not limited to. As used throughout this application, thesingular forms “a,” “an,” and “the” include plural referents unless thecontent explicitly indicates otherwise. Thus, for example, reference to“an element” or “an element” includes a combination of two or moreelements, notwithstanding use of other terms and phrases for one or moreelements, such as “one or more.” The term “or” is, unless indicatedotherwise, non-exclusive, i.e., encompassing both “and” and “or.” Termsdescribing conditional relationships, e.g., “in response to X, Y,” “uponX, Y,”, “if X, Y,” “when X, Y,” and the like, encompass causalrelationships in which the antecedent is a necessary causal condition,the antecedent is a sufficient causal condition, or the antecedent is acontributory causal condition of the consequent, e.g., “state X occursupon condition Y obtaining” is generic to “X occurs solely upon Y” and“X occurs upon Y and Z.” Such conditional relationships are not limitedto consequences that instantly follow the antecedent obtaining, as someconsequences may be delayed, and in conditional statements, antecedentsare connected to their consequents, e.g., the antecedent is relevant tothe likelihood of the consequent occurring. Statements in which aplurality of attributes or functions are mapped to a plurality ofobjects (e.g., one or more processors performing steps A, B, C, and D)encompasses both all such attributes or functions being mapped to allsuch objects and subsets of the attributes or functions being mapped tosubsets of the attributes or functions (e.g., both all processors eachperforming steps A-D, and a case in which processor 1 performs step A,processor 2 performs step B and part of step C, and processor 3 performspart of step C and step D), unless otherwise indicated. Further, unlessotherwise indicated, statements that one value or action is “based on”another condition or value encompass both instances in which thecondition or value is the sole factor and instances in which thecondition or value is one factor among a plurality of factors. Unlessotherwise indicated, statements that “each” instance of some collectionhave some property should not be read to exclude cases where someotherwise identical or similar members of a larger collection do nothave the property, i.e., each does not necessarily mean each and every.Limitations as to sequence of recited steps should not be read into theclaims unless explicitly specified, e.g., with explicit language like“after performing X, performing Y,” in contrast to statements that mightbe improperly argued to imply sequence limitations, like “performing Xon items, performing Y on the X'ed items,” used for purposes of makingclaims more readable rather than specifying sequence. Statementsreferring to “at least Z of A, B, and C,” and the like (e.g., “at leastZ of A, B, or C”), refer to at least Z of the listed categories (A, B,and C) and do not require at least Z units in each category. Unlessspecifically stated otherwise, as apparent from the discussion, it isappreciated that throughout this specification discussions utilizingterms such as “processing,” “computing,” “calculating,” “determining” orthe like refer to actions or processes of a specific apparatus, such asa special purpose computer or a similar special purpose electronicprocessing/computing device. Features described with reference togeometric constructs, like “parallel,” “perpendicular/orthogonal,”“square”, “cylindrical,” and the like, should be construed asencompassing items that substantially embody the properties of thegeometric construct, e.g., reference to “parallel” surfaces encompassessubstantially parallel surfaces. The permitted range of deviation fromPlatonic ideals of these geometric constructs is to be determined withreference to ranges in the specification, and where such ranges are notstated, with reference to industry norms in the field of use, and wheresuch ranges are not defined, with reference to industry norms in thefield of manufacturing of the designated feature, and where such rangesare not defined, features substantially embodying a geometric constructshould be construed to include those features within 15% of the definingattributes of that geometric construct. The terms “first”, “second”,“third,” “given” and so on, if used in the claims, are used todistinguish or otherwise identify, and not to show a sequential ornumerical limitation. As is the case in ordinary usage in the field,data structures and formats described with reference to uses salient toa human need not be presented in a human-intelligible format toconstitute the described data structure or format, e.g., text need notbe rendered or even encoded in Unicode or ASCII to constitute text;images, maps, and data-visualizations need not be displayed or decodedto constitute images, maps, and data-visualizations, respectively;speech, music, and other audio need not be emitted through a speaker ordecoded to constitute speech, music, or other audio, respectively.Computer implemented instructions, commands, and the like are notlimited to executable code and can be implemented in the form of datathat causes functionality to be invoked, e.g., in the form of argumentsof a function or API call.

In this patent, to the extent any U.S. patents, U.S. patentapplications, or other materials (e.g., articles) have been incorporatedby reference, the text of such materials is only incorporated byreference to the extent that no conflict exists between such materialand the statements and drawings set forth herein. In the event of suchconflict, the text of the present document governs, and terms in thisdocument should not be given a narrower reading in virtue of the way inwhich those terms are used in other materials incorporated by reference.

Some embodiments may include text navigation indicators or other contextvalues to indicate where one or more document type values is located ina document or verify the validity of an obtained document type value.For example, some embodiments may use a document classifier to determinethat a first document has a document type of “Type01A” and acorresponding document feature of “effective date.” The value for thedocument feature “effective date” may be stored in a first text sectionhaving the header “date listing.” Some embodiments may perform one ormore operations described in this disclosure on the first text sectionto retrieve a corresponding document feature value for the documentfeature “effective date” from the first text section using a SiameseBERT model. As described elsewhere in this disclosure, the relativesparsity of contract data may make dual input learning models such asthe Siamese BERT model useful.

Some embodiments may apply a hierarchical set of deep learning models toobtain document feature values or other values of a document based ontheir corresponding types of values. For example, after using a firstdeep learning model to obtain a first set of document feature values,some embodiments may use a second deep learning model to obtain a secondset of document feature values. The first set of document feature valuesmay be a set of quantities, and the second set of document featurevalues may be a set of strings. Furthermore, some embodiments may obtaina set of pre-trained deep learning models using transfer learningtechniques, where the set of pre-trained deep learning models may befurther updated based on new feedback indicating whether one or morepredicted values is correct.

As described elsewhere in this disclosure, some embodiments may uselearning model parameters that are cached on a local computing device toperform one or more data extraction operations. For example, someembodiments may push a set of learning model parameters to a usercomputing device and then, after receiving a request to perform a dataextraction operation, some embodiments may instruct the user computingdevice to extract data from a document based on the set of learningmodel parameters. Alternatively, or in addition, some embodiments mayuse learning model parameters that are stored in a fast cache, where thespeed of data access with respect to the cash may be greater relative toa speed of data access for a persistent storage used to store documents.For example, some embodiments may obtain model parameters from a Redisdatastore.

In some embodiments, operations of the process 400 may includedetermining whether a set of criteria is satisfied based on the set ofdocument type values, as indicated for block 420. In some embodiments,one or more rules may be used to verify or otherwise validate fieldvalues obtained from a document. For example, some embodiments may applya first set of rules associated with a first document type to a set ofobtained field values, where the first set of rules. Some embodimentsmay apply a plurality of rules based on a document type category. Forexample, based on a determination that a document has a document typecategory of “transaction contract,” some embodiments may apply a set ofrules to check whether a corresponding set of obtained document featurevalues includes values that populate a first field titled “supplier,” asecond field titled “payment,” or other fields of a corresponding set ofdocument features. Various other rules may be applied, such as a rulethat a set of compliance checks are satisfied, a set of conditionsindicating non-suspicious activity is satisfied.

In some embodiments, different sets of criteria may be applied. Forexample, a first set of criteria may include a set of criteriaindicating data quality, where criteria may include determining whethera first set of fields titled “required” has been populated with anassociated set of field values or whether a set of derived field valueshave been determined. The set of criteria may also include determiningwhether the set of field values is cross-validated with other data,whether a set of ranges associated with one or more fields satisfy acorresponding range threshold, whether the source of the document beinganalyzed is listed as an accepted data source, or the like. A second setof rules may include a set of compliance rules corresponding with anaccount record or a group of account records identifying or otherwiseassociated with an organization. For example, the second set of rulesmay determine whether the field values indicate duplicate payment orinvoices, duplicate purchase numbers for a same item, or the like. Insome embodiments the set of rules being applied to a document may bespecific to a document type, where a determination may be made aboutwhether a document is actually of the document type (e.g., a document isidentified by a set of headers or vocabulary as a contract), whether adocument identifier is correct, whether a set of document feature valuesis present or within a range of known acceptable values, or the like.For example, some embodiments may determine whether a document hasfeatures associated with a document type “contract,” whether thedocument has a correct identifier, whether the document lists either nodiscount or an incorrect discount value, or the like. Some embodimentsmay further determine reconciliation rules, where the reconciliationrules may include determining mismatches in quantity, price, or othernumeric values.

Certain embodiments include a method for storing values in associationwith an unstructured natural language document. The method including:obtaining an unstructured natural language document including one ormore text sections; determining a count of n-grams of the unstructurednatural language document; determining, based on the count of n-grams ofthe unstructured natural language document, a vector for the document;determining, based on the vector using a first machine learning model, acategory for the document; retrieving, based the category, a set offeatures for the document; determining, based on the set of features andtext of the unstructured natural language document with a second machinelearning model, a score for a text section of the unstructured naturallanguage document; selecting, based on the score, the text section;obtaining, based on the selection of text section, a feature value; andstoring the feature value in association with the unstructured naturallanguage document. In some embodiments, the first machine learning modelincludes a first deep learning model, and the second machine learningmodel includes a second deep learning model. In some embodiments, thesecond machine learning model includes a multi-channel transformer, anddetermining the score includes: providing the set of features to a firstchannel of the multi-channel transformer; and providing the text of theunstructured natural language document to a second channel of themulti-channel transformer. In some embodiments, the unstructured naturallanguage document includes a contract document outlining contract terms,and the feature value includes a numerical value for one or more of thecontract terms, with the method further including generating, based onthe contract term, a second contract, wherein the second contractincludes a self-executing contract including computer code defining thecontract terms and configured to be executed to enforce the contractterms.

1. A method of natural language processing (NLP) of an unstructurednatural language document, the method comprising: obtaining, by anatural language processing (NLP) computer system, an unstructurednatural language document, the unstructured natural language documentcomprising text sections comprising sequences of textual elements;determining, by the NLP computer system, n-grams corresponding to thesequences of textual elements; determining, by the NLP computer system,a count of the n-grams, the count of the n-grams comprising, for each ofdifferent n-grams, a count of the number of times the n-gram appears inthe sequences of textual elements; determining, by the NLP computersystem and based on the count of n-grams, a vector of the unstructurednatural language document; determining, by the NLP computer system andbased on the vector using a first deep learning model, a category forthe unstructured natural language document; obtaining, by the NLPcomputer system in response to determining the category and from adatabase storing sets of document features associated with documentcategories, a set of document features associated with the category;determining, by the NLP computer system in response to determining theset of document features and using a second deep learning model, a scorefor a text section of the unstructured natural language document basedon the set of features and text of the unstructured natural languagedocument; selecting, by the NLP computer system based on the score, thetext section; determining, by the NLP computer system based on the textsection, a feature value; storing, by the NLP computer system in adatabase, the feature value in association with the unstructured naturallanguage document, wherein the unstructured natural language documentcomprises a contract document outlining contract terms, and wherein oneor more of the contract terms are defined by the feature value; andgenerating, by the NLP computer system based on the contract terms, asecond contract, wherein the second contract comprises a self-executingcontract comprising computer code comprising conditional statementsdefining the contract terms and configured to be executed to enforce thecontract terms, and wherein generating the second contract comprises:generating the computer code comprising the conditional statementsdefining the contract terms; and storing, on a distributed ledgerpeer-to peer-network, the computer code, wherein the computer codestored on the distributed ledger peer-to peer-network is configured tobe executed to enforce the conditional statements defining the contractterms.
 2. The method of claim 1, the method further comprising:determining, by the NLP computer system, a micro-ai-model for a featureof the set of features, wherein the score for the text section of theunstructured natural language document is determined using themicro-ai-model for the feature.
 3. The method of claim 1, wherein thesecond deep learning model comprises a multi-channel transformer model,and wherein determining the score comprises: providing, to a firstchannel of the multi-channel transformer, the set of document features;and providing, to a second channel of the multi-channel transformer, thetextual elements of the unstructured natural language document, whereinthe multi-channel transformer determines the score for the text sectionbased on the first channel processing the set of document features andthe second channel processing the textual elements.
 4. The method ofclaim 1, wherein the feature values comprises a numerical value. 5.(canceled)
 6. (canceled)
 7. The method of claim 1, wherein the contractcomprises a self-executing contract specifying an exchange of monetaryfunds in response to occurrence of an event, and wherein the computercode stored on the distributed ledger peer-to peer-network is executedto cause the exchange of the monetary funds in response to occurrence ofthe event.
 8. A system for natural language processing (NLP) of anunstructured natural language document, the system comprising: acomputer processor; and non-transitory computer readable storage mediumcomprising program instructions stored thereon that are executable bythe computer processor to cause the following operations: obtaining, bya natural language processing (NLP) computer system, an unstructurednatural language document, the unstructured natural language documentcomprising text sections comprising sequences of textual elements;determining, by the NLP computer system, n-grams corresponding to thesequences of textual elements; determining, by the NLP computer system,a count of the n-grams, the count of the n-grams comprising, for each ofdifferent n-grams, a count of the number of times the n-gram appears inthe sequences of textual elements; determining, by the NLP computersystem and based on the count of n-grams, a vector of the unstructurednatural language document; determining, by the NLP computer system andbased on the vector using a first deep learning model, a category forthe unstructured natural language document; obtaining, by the NLPcomputer system in response to determining the category and from adatabase storing sets of document features associated with documentcategories, a set of document features associated with the category;determining, by the NLP computer system in response to determining theset of document features and using a second deep learning model, a scorefor a text section of the unstructured natural language document basedon the set of features and text of the unstructured natural languagedocument; selecting, by the NLP computer system based on the score, thetext section; determining, by the NLP computer system based on the textsection, a feature value; storing, by the NLP computer system in adatabase, the feature value in association with the unstructured naturallanguage document, wherein the unstructured natural language documentcomprises a contract document outlining contract terms, and one or moreof the contract terms are defined by the feature value; and generating,by the NLP computer system based on the contract terms, a secondcontract, wherein the second contract comprises a self-executingcontract comprising computer code comprising conditional statementsdefining the contract terms and configured to be executed to enforce thecontract terms, and wherein generating the second contract comprises:generating the computer code comprising the conditional statementsdefining the contract terms; and storing, on a distributed ledgerpeer-to peer-network, the computer code, wherein the computer codestored on the distributed ledger peer-to peer-network is configured tobe executed to enforce the conditional statements defining the contractterms.
 9. The system of claim 8, the operations further comprising:determining, by the NLP computer system, a micro-ai-model for a featureof the set of features, wherein the score for the text section of theunstructured natural language document is determined using themicro-ai-model for the feature.
 10. The system of claim 8, wherein thefeature values comprises a numerical value.
 11. (canceled)
 12. Anon-transitory computer readable storage medium comprising programinstructions stored thereon that are executable by the computerprocessor to cause the following operations for natural languageprocessing (NLP) of an unstructured natural language document:obtaining, by a natural language processing (NLP) computer system, anunstructured natural language document, the unstructured naturallanguage document comprising text sections comprising sequences oftextual elements; determining, by the NLP computer system, n-gramscorresponding to the sequences of textual elements; determining, by theNLP computer system, a count of the n-grams, the count of the n-gramscomprising, for each of different n-grams, a count of the number oftimes the n-gram appears in the sequences of textual elements;determining, by the NLP computer system and based on the count ofn-grams, a vector of the unstructured natural language document;determining, by the NLP computer system and based on the vector using afirst deep learning model, a category for the unstructured naturallanguage document; obtaining, by the NLP computer system in response todetermining the category and from a database storing sets of documentfeatures associated with document categories, a set of document featuresassociated with the category; determining, by the NLP computer system inresponse to determining the set of document features and using a seconddeep learning model, a score for a text section of the unstructurednatural language document based on the set of features and text of theunstructured natural language document; selecting, by the NLP computersystem based on the score, the text section; determining, by the NLPcomputer system based on the text section, a feature value; and storing,by the NLP computer system in a database, the feature value inassociation with the unstructured natural language document, wherein theunstructured natural language document comprises a contract documentoutlining contract terms, and one or more of the contract terms aredefined by the feature value; generating, by the NLP computer systembased on the contract terms, a second contract, wherein the secondcontract comprises a self-executing contract comprising computer codecomprising conditional statements defining the contract terms andconfigured to be executed to enforce the contract terms, and whereingenerating the second contract comprises: generating the computer codecomprising the conditional statements defining the contract terms; andstoring, on a distributed ledger peer-to peer-network, the computercode, wherein the computer code stored on the distributed ledger peer-topeer-network is configured to be executed to enforce the conditionalstatements defining the contract terms.
 13. The medium of claim 12, theoperations further comprising: determining, by the NLP computer system,a micro-ai-model for a feature of the set of features, wherein the scorefor the text section of the unstructured natural language document isdetermined using the micro-ai-model for the feature.
 14. The medium ofclaim 12, wherein the feature values comprises a numerical value. 15.(canceled)
 16. (canceled)
 17. (canceled)
 18. (canceled)
 19. (canceled)20. (canceled)
 21. (canceled)
 22. The method of claim 1, whereindetermining the vector of the unstructured natural language documentcomprises determining, using term frequency—inverse document frequency(TF-IDF) scoring based on the count of n-grams, the vector of theunstructured natural language document.
 23. The system of claim 8,wherein determining the vector of the unstructured natural languagedocument comprises determining, using term frequency—inverse documentfrequency (TF-IDF) scoring based on the count of n-grams, the vector ofthe unstructured natural language document.
 24. The medium of claim 12,wherein determining the vector of the unstructured natural languagedocument comprises determining, using term frequency—inverse documentfrequency (TF-IDF) scoring based on the count of n-grams, the vector ofthe unstructured natural language document.
 25. The method of claim 1,further comprising: determining a first feature associated with theunstructured natural language document; determining a second featureassociated with the unstructured natural language document; determining,by the NLP computer system, a first micro-ai-model corresponding to thefirst feature; determining, by the NLP computer system, a secondmicro-ai-model corresponding to the second feature, wherein the scorefor the text section of the unstructured natural language document isdetermined based on application of the first micro-ai-model to the textsection of the unstructured natural language document; determining, bythe NLP computer system based on application of the secondmicro-ai-model to a second text section of the unstructured naturallanguage document, a second score; selecting, by the NLP computer systembased on the second score, the second text section; and determining, bythe NLP computer system based on the second text section, a secondfeature value, wherein one or more of the contract terms are defined bythe second feature value.
 26. The system of claim 8, the operationsfurther comprising: determining a first feature associated with theunstructured natural language document; determining a second featureassociated with the unstructured natural language document; determining,by the NLP computer system, a first micro-ai-model corresponding to thefirst feature; determining, by the NLP computer system, a secondmicro-ai-model corresponding to the second feature, wherein the scorefor the text section of the unstructured natural language document isdetermined based on application of the first micro-ai-model to the textsection of the unstructured natural language document; determining, bythe NLP computer system based on application of the secondmicro-ai-model to a second text section of the unstructured naturallanguage document, a second score; selecting, by the NLP computer systembased on the second score, the second text section; and determining, bythe NLP computer system based on the second text section, a secondfeature value, wherein one or more of the contract terms are defined bythe second feature value.
 27. The medium of claim 12, the operationsfurther comprising: determining a first feature associated with theunstructured natural language document; determining a second featureassociated with the unstructured natural language document; determining,by the NLP computer system, a first micro-ai-model corresponding to thefirst feature; determining, by the NLP computer system, a secondmicro-ai-model corresponding to the second feature, wherein the scorefor the text section of the unstructured natural language document isdetermined based on application of the first micro-ai-model to the textsection of the unstructured natural language document; determining, bythe NLP computer system based on application of the secondmicro-ai-model to a second text section of the unstructured naturallanguage document, a second score; selecting, by the NLP computer systembased on the second score, the second text section; and determining, bythe NLP computer system based on the second text section, a secondfeature value, wherein one or more of the contract terms are defined bythe second feature value.
 28. The system of claim 8, wherein thecontract comprises a self-executing contract specifying exchange ofmonetary funds in response to occurrence of an event, and wherein thecomputer code stored on the distributed ledger peer-to peer-network isexecuted to cause the exchange of the monetary funds in response tooccurrence of the event
 29. The medium of claim 12, wherein the contractcomprises a self-executing contract specifying exchange of monetaryfunds in response to occurrence of an event, and wherein the computercode stored on the distributed ledger peer-to peer-network is executedto cause the exchange of the monetary funds in response to occurrence ofthe event.